Automotive color matching system and method

ABSTRACT

A computer system for identifying coating colors using a digital image comprises one or more processors and one or more computer-readable media having store thereon executable instructions that when executed by the one or more processors configure the computer system to perform various actions. For example, the computer system can receive, through the network connection, a user-provided digital image of a vehicle. The computer system can also identify, with an image processing module, one or more vehicle characteristics within the user-provided digital image. Further, the computer system can identify at least one coating color based on the one or more vehicle characteristics.

BACKGROUND

When a vehicle undergoes repair, a repair paint is applied to thevehicle, which should match the original paint. However, due to colorshifts in the original paint applied to vehicles during manufacturing,it is difficult to match the repair paint to the original paint.Differences between the original vehicle paint and a repair paint on thevehicle can be perceived. The color variations of paint produced byoriginal equipment manufacturers are difficult to color match in themultitude of auto body repair shops that repaint vehicles.

Vehicles typically include one or more identification tags, including acolor code that refers to the original paint formulation. Auto bodyrepair shop employees are conventionally required to hand-enter metadataassociated with the vehicle in the repair shop in order to identify thecolor code that best matches the paint of a vehicle undergoing repair.The metadata includes vehicle make, model, year, color code, VIN, etc.However, hand-entry is prone to mistakes, cumbersome, time-consuming.Further, color codes are becoming more and more difficult to locate onvehicle bodies, making it labor-intensive for repair shop employees tofind and enter color code data. Delays associated with refinish paintcolor matching in the repair process are costly to the auto body repairshop in terms of productivity and associated expenses.

Accordingly, there are many opportunities for new systems and methodsthat aid repair shops in their selection of a paint color.

BRIEF SUMMARY

A computer system for identifying coating colors using a digital imagecomprises one or more processors and one or more computer-readable mediahaving store thereon executable instructions that when executed by theone or more processors configure the computer system to perform variousactions. For example, the computer system can receive, through thenetwork connection, a user-provided digital image of a vehicle. Thecomputer system can also access, within a vehicle template database, oneor more vehicle templates. Further, the computer system can map at leastone conforming vehicle template to the vehicle within the user-provideddigital image, wherein the at least one conforming vehicle templatecomprises associated metadata comprising the one or more vehiclecharacteristics, including one or more associated color codes. Thecomputer system can also identify, with an image processing module, acolor value associated with the vehicle within the user-provided digitalimage. Finally, the computer system calculate a closest match foridentified color value associated with the vehicle from the one or moreassociated color codes.

A computerized method for use with a computer system comprising one ormore processors and one or more computer-readable media having storedthereon executable instructions that when executed by the one or moreprocessors configure the computer system to perform a method ofidentifying coating colors using a digital image. The method cancomprise receiving, through the network connection, a user-provideddigital image of a vehicle. The method can also comprise accessing,within a vehicle template database, one or more vehicle templates. Themethod can further comprise mapping at least one conforming vehicletemplate to the vehicle within the user-provided digital image, whereinthe at least one conforming vehicle template comprises associatedmetadata comprising the one or more vehicle characteristics, includingone or more associated color codes. Also, the method can includeidentifying, with an image processing module, a color value associatedwith the vehicle within the user-provided digital image. The method cancomprise calculating a closest match for identified color valueassociated with the vehicle from the one or more associated color codes.Finally, the method can comprise providing a user the calculated closestmatch.

A computer program product comprising one or more computer storage mediahaving stored thereon computer-executable instructions that, whenexecuted at a processor, cause the computer system to perform a methodfor identifying coating colors using a digital image. The method cancomprise receiving, through a network connection, a user-provideddigital image of a vehicle. The method can also include accessing,within a vehicle template database, one or more vehicle templates. Themethod can further comprise mapping at least one conforming vehicletemplate to the vehicle within the user-provided digital image, whereinthe at least one conforming vehicle template comprises associatedmetadata comprising the one or more vehicle characteristics, includingone or more associated color codes. Also, the method can includeidentifying, with an image processing module, a color value associatedwith the vehicle within the user-provided digital image. The method cancomprise calculating a closest match for identified color valueassociated with the vehicle from the one or more associated color codes.Additionally, the method can include providing a user the calculatedclosest match. Further, the method can comprise receiving user feedbackthat the calculated closest match is incorrect, calculating a colorshift profile, and applying the color shift profile to user-provideddigital images of vehicles comprising camera and lightingcharacteristics of the user-provided digital image of the vehicle.

Additional features and advantages will be set forth in the descriptionwhich follows, and in part will be obvious from the description, or maybe learned by practice. The features and advantages may be realized andobtained by means of the instruments and combinations particularlypointed out in the appended claims. These and other features will becomemore fully apparent from the following description and appended claims,or may be learned by the practice of the examples as set forthhereinafter.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to describe the manner in which the above recited and otheradvantages and features can be obtained, a more particular descriptionbriefly described above will be rendered by reference to specificexamples thereof, which are illustrated in the appended drawings.Understanding that these drawings are merely illustrative and are nottherefore to be considered to be limiting of its scope, the computersystem for dynamically parsing a digital image to identify coatingcolors will be described and explained with additional specificity anddetail through the use of the accompanying drawings in which:

FIG. 1 depicts a schematic diagram of a network-based system foridentifying coating colors using a digital image;

FIG. 2 depicts an exemplary user-provided digital image of a vehicle;

FIG. 3 depicts an exemplary vehicle template database comprising vehicletemplates;

FIG. 4 depicts the exemplary user-provided digital image shown in FIG. 2, wherein a conforming vehicle template is mapped to the vehicle;

FIG. 5 depicts an exemplary repair template database comprising repairtemplates;

FIG. 6 depicts the exemplary user-provided digital image shown in FIG. 2, wherein a conforming repair template is mapped to the vehicle;

FIG. 7 depicts a modified user-provided digital image; and

FIG. 8 illustrates a flow chart of a series of acts in a method foridentifying a coating color using a digital image.

DETAILED DESCRIPTION

A computer system for identifying coating colors using a digital imagecomprises one or more processors and one or more computer-readable mediahaving store thereon executable instructions that when executed by theone or more processors configure the computer system to perform variousactions. For example, the computer system can receive, through thenetwork connection, a user-provided digital image of a vehicle. Thecomputer system can also access, within a vehicle template database, oneor more vehicle templates. Further, the computer system can map at leastone conforming vehicle template to the vehicle within the user-provideddigital image, wherein the at least one conforming vehicle templatecomprises associated metadata comprising the one or more vehiclecharacteristics, including one or more associated color codes. As usedherein, “vehicle characteristics” may also comprise a vehicle make,model, year, or vehicle identification number (VIN). The computer systemcan also identify, with an image processing module, a color valueassociated with the vehicle within the user-provided digital image.Finally, the computer system calculate a closest match for identifiedcolor value associated with the vehicle from the one or more associatedcolor codes.

As such, the computer system can provide several benefits to the art.For example, the described coating color identification process mayreduce the chance of human error when entering vehicle characteristics.The color identification process may also detect subtle color variancesthat are not detectable by the human eye. Further the described computersystem may increase the speed at which auto body repair shops identify arefinish paint color thereby increasing their productivity.

Turning now to the figures, FIG. 1 illustrates a schematic of acomputerized system for identifying a coating color using a digitalimage. As shown, a computer system 100 is in communication with coatingcolor analysis software 105 through a network connection 110. Oneskilled in the art will appreciate that the depicted schematic is merelyexemplary, and although the computer system 100 is depicted in FIG. 1 asa mobile phone, the computer system 100 can take a variety of forms. Forexample, the computer system 100 may be a laptop computer, a tabletcomputer, a wearable device, a desktop computer, a mainframe, etc. Asused herein, the term “computer system” includes any device, system, orcombination thereof that includes one or more processors, and a physicaland tangible computer-readable memory capable of having thereoncomputer-executable instructions that are executable by the one or moreprocessors.

The one or more processors may comprise an integrated circuit, afield-programmable gate array (FPGA), a microcontroller, an analogcircuit, or any other electronic circuit capable of processing inputsignals. The memory may be physical system memory, which may bevolatile, non-volatile, or some combination of the two. The term“memory” may also be used herein to refer to non-volatile mass storagesuch as physical storage media. Examples of computer-readable physicalstorage media include RAM, ROM, EEPROM, solid state drives (“SSDs”),flash memory, phase-change memory (“PCM”), optical disk storage,magnetic disk storage or other magnetic storage devices, or any otherhardware storage device(s). The computer system 100 may be distributedover a network environment and may include multiple constituent computersystems.

The computer system 100 can comprise one or more computer-readablestorage media having stored thereon executable instructions that whenexecuted by the one or more processors configure the computer system 100to execute the coating color analysis software 105. The coating coloranalysis software 105 may comprise various modules, such as an interfacemodule 120 and an image processing module 125. As used herein, a modulemay comprise a software component, including a software object, ahardware component, such as a discrete circuit, a FPGA, a computerprocessor, or some combination of hardware and software.

One will understand, however, that separating modules into discreteunits is at least somewhat arbitrary and that modules can be combined,associated, or separated in ways other than shown in FIG. 1 and stillaccomplish the purposes of the computer system. Accordingly, the modules120 and 125 of FIG. 1 are only shown for illustrative and exemplarypurposes.

The coating color analysis software 105 may also be in communicationwith one or more databases. For example, the coating color analysissoftware 105 may be in communication with a vehicle template database130, a vehicle identification number (“VIN”) database 135, and a repairtemplate database 140. As used herein, a database may comprise locallystored data, remotely stored data, data stored within an organized datastructure, data stored within a file system, or any other stored datathat is accessible to the coating color analysis software 105.

The coating color analysis software 105 may be configured to receive auser-provided digital image of a vehicle 115. For example, the user mayuse the computer system 100 to upload a user-provided digital image 115to the coating color analysis software 105 via the network connection110. As used herein, a digital image may comprise a photograph (e.g., astill of a physical reality) or and an image either digitallyrepresenting reality or a digitally-created artifact. The interfacemodule 120 may provide an interface for selecting a digital imageavailable to the user and uploading the user-provided digital image 115into the coating color analysis software 105. Additionally oralternatively, the interface module 120 may allow the user to provideadditional or alternative vehicle-identifying data. For example, theinterface module 120 may allow the user to type, speak, or otherwiseidentify details about the vehicle (e.g., make, model, color, etc.).

The interface module 120 may be configured to receive user-derived audiocomprising vehicle characteristics and translate the user-derived audioto machine-encoded text. For example, the coating color analysissoftware 105 may be configured to receive user-derived audio from avoice recognition system installed on the computer system 100.Additionally or alternatively, the voice recognition system maytranslate the user-derived audio to machine-encoded text before sendingthe machine-encoded text to the coating color analysis software 105.

The user-provided digital image 115 may comprise a color or black/whitephotograph of the vehicle showing at least a portion of the body of thevehicle. The interface module 120 may provide an interface foridentifying the angle at which the user-provided digital image 115 wastaken. The interface module 120 may also provide an interface foruploading more than one image of the vehicle at multiple angles. Theimage processing module 125 may be configured to identify theobservation angle of the user-provided digital image 115 without inputfrom the user.

As shown in FIG. 1 , the interface module 120 may be in communicationwith the image processing module 125 and configured to send theuser-provided digital image 115 to the image processing module 125. Theimage processing module 125 may access, within the vehicle templatedatabase 130, one or more vehicle templates, and map a conformingvehicle template to the vehicle within the user-provided digital image115. The conforming vehicle template may comprise associated metadatacomprising the one or more vehicle characteristics, including one ormore associated color codes.

A vehicle template may comprise a digital description of the physical,viewable characteristics of a particular vehicle. In some cases, thevehicle templates may comprise labelled data relating to vehicles thatcan be loaded into a neural network. A vehicle template may also beassociated with metadata that describes various aspects of theunderlying vehicle. The metadata may include vehicle characteristicssuch as model, make, body style, year range, color, and one or moreassociated color codes. After the conforming vehicle template is mappedto the vehicle within the user-provided digital image 115, the imageprocessing module 125 may identify vehicle characteristics based on themetadata associated with the conforming vehicle template.

The vehicle templates may comprise line drawings of vehicles. Thevehicle templates may also comprise three-dimensional models ofvehicles. The image processing module 125, therefore, may map theconforming vehicle template to the vehicle in the user-provided digitalimage 115 by line matching. The image processing module 125 may also beconfigured to automatically adjust the size of the conforming vehicletemplate to align with the vehicle in the user-provided digital image115. Therefore, a vehicle template may be conformed by matching variousvehicle templates and choosing the vehicle template with the leastdifferences (e.g., structural differences) through line matching.

The vehicle templates may also comprise a color element, and the imageprocessing module 125 may be configured to determine the color of thevehicle in the user-provided digital image 115 and map a color-matchedconforming vehicle template to the vehicle. Therefore, thecolor-matching of the conforming vehicle template additionally matchesthe determined color of the vehicle in the user-provided digital image115 with color information of the vehicle template, wherein a color ismatched if the quantitative difference in a specific color space such as“delta e (ΔE) is less than X”.

Additionally or alternatively, the image processing module 125 maycomprise a machine learning algorithm that is configured to identifyvehicle characteristics within the user-provided digital image 115. Themachine learning algorithm may be taught using annotated vehicletemplates stored within the vehicle template database 130. In somecases, the machine learning algorithm may also map the identifiedvehicle to vehicle templates within the vehicle template database 130.The machine learning algorithm may comprise any number of differentobject recognition and object classification algorithms, including aconvolutional neural network. Information can then be gathered from themetadata associated with the vehicle template.

The vehicle template database 130 may include database subsets that areorganized based on information provided by the user. For example, if theuser indicated that the vehicle is a TOYOTA, the vehicle templatedatabase 130 may include a database subset with vehicle templatesspecific to TOYOTA vehicles. Additionally or alternatively, the imageprocessing module 125 may first determine the color of the vehicle inthe user-provided digital image 115 and the vehicle template database130 may include a database subset with vehicle templates specific to theidentified color. For example, a particular car model may come in eightdifferent factory colors. The vehicle template database 130 may comprisevehicle templates that car model in each of the eight different factorycolors.

Additionally, the image processing module 125 may identify a color valueassociated with the vehicle within the user-provided digital image 115.The color value associated with the vehicle may be an RGB value.Additionally or alternatively, the color value may be a color family(e.g., white, red, blue, silver). The image processing module may alsocalculate a closest match for identified color value associated with thevehicle from the one or more associated color codes. The closest matchmay be determined by comparing the quantitative differences in aspecific color space such as “delta e (ΔE) is less than X”. The closestmatch may represent the most probable color code for the car.

As shown in FIG. 1 , The image processing module 125 may be incommunication with the computer system 100 through the networkconnection 110 and configured to send the computer system 100 theclosest match 145. The Prime, Variants, and Specials for the closestmatch 145 may also be sent to the computer system 100. Additionally oralternatively, the interface module 120 may be configured to display theidentified vehicle characteristics based on the metadata associated withthe conforming vehicle template.

Additionally or alternatively, the image processing module may include asmart learning color verification process. For example, the interfacemodule 120 or image processing module 125 may be configured to receiveuser feedback from the computer system 100 that the closest match colorcode is incorrect. The interface module 120 or image processing module125 may be further configured to receive from the user an indication ofthe correct color code for the vehicle. Based on variance between theincorrect closest match color code and the user-identified color code,the image processing module may calculate a color shift profile. Theimage processing module 125 may apply the color shift profile tosubsequent user-provided digital images comprising the camera andlighting characteristics of the user-provided digital image 115. Theimage processing module 125 may make alternative or additionalmodifications to a user-provided digital image before calculating aclosest match 145.

Additionally or alternatively, the image processing module 125 maydetermine if the vehicle 115 has been repainted. For example, the imageprocessing module 125 may be configured to identify when the closestmatch falls outside a predetermined threshold, such as “delta e (ΔE) isless than X”. If the closest match is identified as falling outside thatthreshold, the image processing module 125 may be configured to send thecomputer system 100 an indication that the vehicle was likely repainted.As discussed above, the image processing module 125 may be configured tosend the computer system 100 confidence information based on thequantitative difference between the identified color value and colorcode.

The user-provided digital image 115 may additionally or alternativelycomprise a photograph of image text associated with the vehicle. Theimage text may include the VIN, color code, make, model, and/or year ofthe vehicle. The image processing module 125 may be configured toidentify image text within the user-provided digital image 115, andthereafter translate the identified image text to machine-encoded textusing optical character recognition technology. If the image textcomprises the VIN, the image processing module 125 may be incommunication with the VIN database 135. The image processing module 125may therefore use the machine-encoded text to search the VIN database135 and identify vehicle characteristics. Alternatively, the vehicletemplate database 130 may comprise a VIN look-up table.

The VIN database 135 may comprise multiple look-up tables thatcorrespond to specific letters or numbers in the VIN. For example, theVIN database may comprise a manufacturer look-up table. The imageprocessing module 125 may use the second and third digits in the VIN tosearch the manufacturer look-up table and identify the manufacturer.Similarly, the VIN database may comprise a vehicle descriptor look-uptable. The image processing module 125 may use the fourth through eighthdigits of the VIN to search the vehicle descriptor look-up table andidentify the brand, engine size, and type of vehicle. Additionally oralternatively, the image processing module may use a machine learningalgorithm to identify vehicle characteristics based on the VIN. Themachine learning algorithm may be taught using annotated VINs storedwithin the VIN database 135. The interface module 120 may be configuredto display to the user the identified vehicle characteristics based onthe VIN.

The VIN database may further comprise a color code look-up table. Theimage processing module 125 may use the identified vehiclecharacteristics to search within the color code look-up table andidentify possible color codes based on the vehicle's make, model, year,etc. For example, the image processing module 125 may identify withinthe color code look-up table that nine possible color codes exist for a2014 TOYOTA COROLLA. If no other information is known about the vehicle,the image processing module 125 may be configured to send all possiblecolor codes to the computer system 100 through the network 110. If theuser identifies the color of the vehicle, or the color of the vehicle isidentified in the user-provided digital image 115, the image processingmodule 125 may filter the possible color codes before sending the colorcodes to the computer system 100. For example, if the user or imageprocessing module 125 identifies that the 2014 TOYOTA COROLLA is white,the image processing module 125 may be configured to send white colorcodes (e.g. 070 for Blizzard/White Pearl Crystal and 040 for SuperWhite) to the computer system color 100.

The image processing module 125 may also be configured to identify arepair area within the user-provided digital image 115. The imageprocessing module 125 may detect a repair area by identifying where themapped vehicle template differs from the vehicle. The image processingmodule 125 may access repair templates within a repair template database140 and map a conforming repair template to the repair area within theuser-provided digital image 115. The repair template database 140 may beorganized into database subsets based on vehicle characteristics orrepair area characteristics.

The repair templates may comprise a digital description of the physical,viewable characteristics of a particular repair. As with the vehicletemplates, repair templates may comprise labelled data relating torepairs that can be loaded into a neural network. The repair templatesmay also be associated with metadata that describes various aspects ofthe underlying repair. The metadata may include detailed instructionsconcerning the repair, the estimated cost of the repair, and theestimated paint usage requirement for the repair.

The repair templates may comprise line drawings of vehicles. The imageprocessing module 125, therefore, may map the conforming repair templateto the repair area in the user-provided digital image 115 by linematching. The image processing module 125 may also be configured toautomatically adjust the size or angle of the conforming repair templateto align with the repair area in the user-provided digital image 115.

Additionally or alternatively, the image processing module 125 maycomprise a machine learning algorithm that is configured to identifyrepair areas within the user-provided digital image 115. The machinelearning algorithm may be taught using annotated repair templates storedwithin the repair template database 140. In some cases, the machinelearning algorithm may also map the identified repair area to repairtemplates within the repair template database 130. The machine learningalgorithm may comprise any number of different object recognition andobject classification algorithms, including a convolutional neuralnetwork. Information can then be gathered from the metadata associatedwith the repair template.

After a conforming repair template is mapped to the repair area withinthe user-provided digital image 115, the image processing module 125 mayidentify repair characteristics 150 based on the metadata associatedwith the conforming repair template. The image processing module 125 maybe in communication with the computer system 100 through the networkconnection 110 and configured to send the computer system 100 theidentified repair characteristics 150. Additionally or alternatively,the interface module 120 may be configured to display the identifiedrepair characteristics 150 based on the metadata associated with theconforming repair template.

Additionally, the image processing module 125 may parse the repair areafrom the user-provided digital image 115 and create a modifieduser-provided digital image 700 (not shown, see FIG. 7 ) by replacingthe parsed at least one repair area with visual data from the conformingvehicle template. The image processing module 125 may be configured tosend the modified user-provided digital image to the computer system 100through the network 110. Additionally or alternatively, the interfacemodule 120 may be configured to show the user the modified user-provideddigital image 700 (not shown, see FIG. 7 ).

FIG. 2 depicts an exemplary user-provided digital image 115. The angleof the user-provided digital image 115 shown in FIG. 2 is merelyexemplary. The user-provided digital image 115 may be taken from anyangle. The user-provided digital image 115 includes a vehicle 200 and arepair area 205.

FIG. 3 depicts a portion of an exemplary vehicle template database 130comprising vehicle templates 300 a-300 c. As shown, the vehicle templatedatabase 130 includes a vehicle template 300 a that corresponds to thecolor and shape of the vehicle 200 shown in FIG. 1 . The vehicletemplate database 130 also includes a vehicle template 300 b thatcomprises the same shape of the vehicle 200 in FIG. 1 but is not thesame color. A vehicle template 300 c that has neither the correspondingcolor nor shape as the vehicle 200 shown in FIG. 2 . Additionally oralternatively, the vehicle template database 130 may be organized invehicle template database subsets based on vehicle characteristics.

FIG. 4 shows the user-provided digital image 115 wherein the conformingvehicle template 300 a has been mapped to the vehicle 200. As describedabove, the image processing module 125 may map the conforming vehicletemplate 300 a by line matching and/or color matching. The imageprocessing module 125 may also be configured to adjust the size of theconforming vehicle template 300 a to align with the vehicle 200.

The conforming vehicle template 300 a may be associated with metadatathat describes various aspects of the underlying vehicle 200. Themetadata may include vehicle characteristics such as model, make, bodystyle, year range, color, and at least one color code. As stated above,the image processing module 125 may be in communication with thecomputer system 100 through the network connection 110 and configured tosend the computer system 100 at least one identified closest match 145.Additionally or alternatively, the interface module 120 may beconfigured to display the identified vehicle characteristics based onthe metadata associated with the conforming vehicle template 300 a.

As shown in FIG. 4 , the repair area 205 is unmapped, as the conformingvehicle template 300 a does not include the same repair area. The imageprocessing module 125 may be configured to detect the repair area 205 byidentifying where the mapped conforming vehicle template 300 a differsfrom the vehicle 200. For instance, the depicted repair area 205 maycomprise a dent in the front, driver's side fender. The vehicle template300 a will not comprise an equivalent dent. As such, the vehicletemplate 300 a can be digitally overlaid onto the user-provided digitalimage 115. A difference calculation can then be performed to identifythat the front, driver's side fender (i.e., the repair area) differsfrom the vehicle template by a predetermined threshold. Thepredetermined threshold may comprise a volume amount, a color amount, aline matching deviation, or a number of other measurements ofdifference.

Upon identifying the existence of a difference at the front, driver'sside fender, the image processing module 125 may access, within a repairtemplate database 150, repair templates, as shown in FIG. 5 . FIG. 5 adepicts a portion of an exemplary repair template database subset withina repair template database 150 comprising repair templates 500 a-500 cthat are specific to the vehicle 200. A repair template 500 a comprisesthe same repair area as the repair area 205 in the user-provided digitalimage 115. Additionally or alternatively, the repair template databasesubset may comprise repair templates specific to an area of a vehicle(e.g., right, front bumper).

FIG. 6 shows a user-provided digital image 115 wherein the conformingrepair template 500 a has been mapped to the repair area 205. Asdescribed above, the image processing module 125 may map the conformingrepair template 500 a by minimizing the difference between a subset ofthe available repair templates. For instance, multiple repair templatesmay exist that comprise damage to the front, driver's side fender. Theimage processing module 125 may compare each of the templates associatedwith damage to the front, driver's side fender to the user-provideddigital image 115 until a closest match is identified. The closest matchmay be identified through by minimizing the difference in volume betweenthe template and the user-provided digital image, by minimizing thedifferences between line matching, or by minimizing any number of othermeasurements of difference. The image processing module 125 may also beconfigured to adjust the size or angle of the conforming repair template500 a to align with the repair area 205.

The conforming repair template 500 a may be associated with metadatathat describes various aspects of the underlying repair. The metadatamay include detailed instructions concerning the repair, the estimatedcost of the repair, and the estimated paint usage requirement for therepair. After a conforming repair template 500 a is mapped to the repairarea 205, the image processing module 125 may identify repaircharacteristics based on the metadata associated with the conformingrepair template 500 a. The image processing module 125 may be incommunication with the computer system 100 through the networkconnection 110 and configured to send the computer system 100 theidentified repair characteristics.

For example, in one case, a lightly damaged front, driver's side fenderis mapped to a conforming repair template 500 a that is associated withparticularly the same, light damage. That conforming repair template 500a is associated with metadata indicating the automotive body filler andpaint can be used to repair the damage. The metadata may furtherindicate an expected cost associated with the minor repair. In contrast,in another case, a heavily damaged front, driver's side fender is mappedto a conforming repair template 500 a that is associated withparticularly the same heavy damage. That conforming repair template 500a is associated with metadata indicating that the entire front, driver'sside fender panel must be replaced and painted to match the rest of thevehicle. The metadata may indicate a relatively higher expected costassociated with the major repair.

One will appreciate that the above example is provided for clarity andsimplicity. The same methods and systems can be applied to damage onother areas of a vehicle. Further, the same methods and systems can beapplied to damage to multiple areas of the vehicle. For instance, avehicle may have damage to the front, driver's side fender, the hood,and the driver's side door panel. In such a case, a repair template thathas damage to these same repair areas 205 may be mapped to the digitalimage of the car. Similarly, multiple different repair templates caneach be mapped to different respective areas of the car. For instance, afirst repair template can be mapped to the front, driver's side fender,a second repair template can be mapped to the hood, and a third repairtemplate can be mapped to the driver's side door panel. The metadataassociated with each template can then be aggregated to identify apotential cost and parts associated with the repair.

FIG. 7 shows a modified user-provided digital image 700 wherein therepair area is parsed from the user-provided digital image 115 andreplaced with visual data from the conforming vehicle template. As shownin FIG. 7 , the modified user-provided digital image 700 comprises thevehicle 200 but does not include the repair area 205. The imageprocessing module 125 may be configured to send the modifieduser-provided digital image 700 to the computer system 100 through thenetwork 110. Additionally or alternatively, the interface module 120 maybe configured to display the modified user-provided digital image 700.

FIG. 8 illustrates a method 800 for identifying coating colors using adigital image. As shown in FIG. 8 , act 805 comprises receiving auser-provided digital image of a vehicle. Act 805 includes receiving,through the network connection, a user-provided digital image of avehicle. For example, as depicted in FIG. 1 , the user may use thecomputer system 100 to upload a user-provided digital image 115 to thecoating color analysis software 105 via the network connection 110. Theinterface module 120 may provide an interface for selecting a digitalimage available to the user and uploading the user-provided digitalimage 115 into the coating color analysis software 105. Additionally oralternatively, the interface module 120 may allow the user to type,speak, or otherwise identify details about the vehicle (e.g., make,model, color, etc.).

The user-provided digital image 115 may comprise a color photograph ofthe vehicle showing at least a portion of the body of the vehicle. Theinterface module 120 may provide an interface for identifying the angleat which the user-provided digital image 115 was taken. The interfacemodule 120 may also provide an interface for uploading more than oneimage of the vehicle at multiple angles. The image processing module 125may be configured to identify the observation angle of the user-provideddigital image 115 without input from the user.

Further, as shown in FIG. 8 , act 810 comprises accessing one or morevehicle templates. Act 805 includes accessing, within a vehicle templatedatabase, one or more vehicle templates. For example, as shown in FIG. 1, the image processing module 125 may access the vehicle templatedatabase 130.

FIG. 8 further shows that act 815 comprises mapping at least oneconforming vehicle template to the vehicle within the user-provideddigital image. Act 815 includes mapping at least one conforming vehicletemplate to the vehicle within the user-provided digital image, whereinthe at least one conforming vehicle template comprises associatedmetadata comprising the one or more vehicle characteristics, includingone or more associated color codes.

For example, as depicted in FIG. 4 , the image processing module 125 maymap a conforming vehicle template 300 a to the vehicle 200 within theuser-provided digital image 115. A vehicle template may comprise adigital description of the physical, viewable characteristics of aparticular vehicle. In some cases, the vehicle templates may compriselabelled data relating to vehicles that can be loaded into a neuralnetwork. A vehicle template may also be associated with metadata thatdescribes various aspects of the underlying vehicle. The metadata mayinclude vehicle characteristics such as model, make, body style, yearrange, color, and one or more associated color codes. After theconforming vehicle template is mapped to the vehicle within theuser-provided digital image 115, the image processing module 125 mayidentify vehicle characteristics based on the metadata associated withthe conforming vehicle template.

The vehicle templates may comprise line drawings of vehicles. Thevehicle templates may also comprise three-dimensional models ofvehicles. The image processing module 125, therefore, may map theconforming vehicle template to the vehicle in the user-provided digitalimage 115 by line matching. The image processing module 125 may also beconfigured to automatically adjust the size of the conforming vehicletemplate to align with the vehicle in the user-provided digital image115. Therefore, a vehicle template may be conformed by matching variousvehicle templates and choosing the vehicle template with the leastdifferences (e.g., structural differences) through line matching.

The vehicle templates may also comprise a color element, and the imageprocessing module 125 may be configured to determine the color of thevehicle in the user-provided digital image 115 and map a color-matchedconforming vehicle template to the vehicle. Therefore, thecolor-matching of the conforming vehicle template additionally matchesthe determined color of the vehicle in the user-provided digital image115 with color information of the vehicle template, wherein a color ismatched if the quantitative difference in a specific color space such as“delta e (ΔE) is less than X”.

Further, act 820 comprises identifying a color value associated with thevehicle within the user-provided digital image. Act 820 includesidentifying, with an image processing module, a color value associatedwith the vehicle within the user-provided digital image. For example,the color value associated with the vehicle may be an RGB value.

As shown in FIG. 8 , act 825 comprises calculating a closest match foridentified color value associated with the vehicle from the one or moreassociated color codes. For example, the closest match may be determinedby comparing the quantitative differences in a specific color space suchas “delta e (ΔE) is less than X”.

Finally, act 830 comprises providing a user the calculated closestmatch. For example, as shown in FIG. 1 , image processing module 125 maybe in communication with the computer system 100 through the networkconnection 110 and configured to send the computer system 100 theclosest match 145. Additionally or alternatively, the interface module120 may be configured to display the identified vehicle characteristicsbased on the metadata associated with the conforming vehicle template.

Although the subject matter has been described in language specific tostructural features and/or methodological acts, it is to be understoodthat the subject matter defined in the appended claims is notnecessarily limited to the described features or acts described above,or the order of the acts described above. Rather, the described featuresand acts are disclosed as example forms of implementing the claims.

The computer system may comprise or utilize a special-purpose orgeneral-purpose computer system that includes computer hardware, suchas, for example, one or more processors and system memory, as discussedin greater detail below. The computer system can also include physicaland other computer-readable media for carrying or storingcomputer-executable instructions and/or data structures. Suchcomputer-readable media can be any available media that can be accessedby a general-purpose or special-purpose computer system.Computer-readable media that store computer-executable instructionsand/or data structures are computer storage media. Computer-readablemedia that carry computer-executable instructions and/or data structuresare transmission media. Thus, by way of example, and not limitation, thecomputer system can comprise at least two distinctly different kinds ofcomputer-readable media: computer storage media and transmission media.

Computer storage media are physical storage media that storecomputer-executable instructions and/or data structures. Physicalstorage media include computer hardware, such as RAM, ROM, EEPROM, solidstate drives (“SSDs”), flash memory, phase-change memory (“PCM”),optical disk storage, magnetic disk storage or other magnetic storagedevices, or any other hardware storage device(s) which can be used tostore program code in the form of computer-executable instructions ordata structures, which can be accessed and executed by a general-purposeor special-purpose computer system to implement the disclosedfunctionality of the computer system.

Transmission media can include a network and/or data links which can beused to carry program code in the form of computer-executableinstructions or data structures, and which can be accessed by ageneral-purpose or special-purpose computer system. A “network” isdefined as one or more data links that enable the transport ofelectronic data between computer systems and/or modules and/or otherelectronic devices. When information is transferred or provided over anetwork or another communications connection (either hardwired,wireless, or a combination of hardwired or wireless) to a computersystem, the computer system may view the connection as transmissionmedia. Combinations of the above should also be included within thescope of computer-readable media.

Further, upon reaching various computer system components, program codein the form of computer-executable instructions or data structures canbe transferred automatically from transmission media to computer storagemedia (or vice versa). For example, computer-executable instructions ordata structures received over a network or data link can be buffered inRAM within a network interface module (e.g., a “NIC”), and theneventually transferred to computer system RAM and/or to less volatilecomputer storage media at a computer system. Thus, it should beunderstood that computer storage media can be included in computersystem components that also (or even primarily) utilize transmissionmedia.

Computer-executable instructions comprise, for example, instructions anddata which, when executed at one or more processors, cause ageneral-purpose computer system, special-purpose computer system, orspecial-purpose processing device to perform a certain function or groupof functions. Computer-executable instructions may be, for example,binaries, intermediate format instructions such as assembly language, oreven source code.

Those skilled in the art will appreciate that the computer system may bepracticed in network computing environments with many types of computersystem configurations, including, personal computers, desktop computers,laptop computers, message processors, hand-held devices, multi-processorsystems, microprocessor-based or programmable consumer electronics,network PCs, minicomputers, mainframe computers, mobile telephones,PDAs, tablets, pagers, routers, switches, and the like. The computersystem may also be practiced in distributed system environments wherelocal and remote computer systems, which are linked (either by hardwireddata links, wireless data links, or by a combination of hardwired andwireless data links) through a network, both perform tasks. As such, ina distributed system environment, a computer system may include aplurality of constituent computer systems. In a distributed systemenvironment, program modules may be located in both local and remotememory storage devices.

Those skilled in the art will also appreciate that the computer systemmay be practiced in a cloud-computing environment. Cloud computingenvironments may be distributed, although this is not required. Whendistributed, cloud computing environments may be distributedinternationally within an organization and/or have components possessedacross multiple organizations. In this description and the followingclaims, “cloud computing” is defined as a model for enabling on-demandnetwork access to a shared pool of configurable computing resources(e.g., networks, servers, storage, applications, and services). Thedefinition of “cloud computing” is not limited to any of the othernumerous advantages that can be obtained from such a model when properlydeployed.

A cloud-computing model can be composed of various characteristics, suchas on-demand self-service, broad network access, resource pooling, rapidelasticity, measured service, and so forth. A cloud-computing model mayalso come in the form of various service models such as, for example,Software as a Service (“SaaS”), Platform as a Service (“PaaS”), andInfrastructure as a Service (“IaaS”). The cloud-computing model may alsobe deployed using different deployment models such as private cloud,community cloud, public cloud, hybrid cloud, and so forth.

A cloud-computing environment may comprise a system that includes one ormore hosts that are each capable of running one or more virtualmachines. During operation, virtual machines emulate an operationalcomputing system, supporting an operating system and perhaps one or moreother applications as well. Each host may include a hypervisor thatemulates virtual resources for the virtual machines using physicalresources that are abstracted from view of the virtual machines. Thehypervisor also provides proper isolation between the virtual machines.Thus, from the perspective of any given virtual machine, the hypervisorprovides the illusion that the virtual machine is interfacing with aphysical resource, even though the virtual machine only interfaces withthe appearance (e.g., a virtual resource) of a physical resource.Examples of physical resources including processing capacity, memory,disk space, network bandwidth, media drives, and so forth.

In view of the foregoing the present computer system relates forexample, without being limited thereto, to the following aspects andconfigurations.

For example, in a first aspect, a computer system for identifyingcoating colors using a digital image can include one or more processors;and one or more computer-readable media having stored thereon executableinstructions that when executed by the one or more processors configurethe computer system to perform at least the following, as in particularperforming the computerized method according to any of the thirteenththrough twenty first aspects: receive, through a network connection, auser-provided digital image of a vehicle; access, within a vehicletemplate database, one or more vehicle templates; map at least oneconforming vehicle template to the vehicle within the user-provideddigital image, wherein the at least one conforming vehicle templatecomprises associated metadata comprising the one or more vehiclecharacteristics, including one or more associated color codes; identify,with an image processing module, a color value associated with thevehicle within the user-provided digital image; and calculate a closestmatch for identified color value associated with the vehicle from theone or more associated color codes.

In a second aspect, in the computer system of the first aspect, thecolor value associated with the vehicle is an RGB value. In a thirdaspect, in the computer system of any of the first or second aspects,the image processing module is configured to determine the color of thevehicle in the user-provided digital image and map a color-matchedconforming vehicle template to the vehicle, wherein the at least onecolor-matched conforming vehicle template comprises associated metadatacomprising the one or more vehicle characteristics. In a fourth aspect,in the computer system of any of the first through third aspects, theexecutable instructions include instructions that are executable toconfigure the computer system to provide a user the associated metadatacomprising the one or more vehicle characteristics. In a fifth aspect,in the computer system of any of the first through fourth aspects, theexecutable instructions include instructions that are executable toconfigure the computer system to: receive user feedback that the closestmatch is incorrect; calculate a color shift profile; and apply the colorshift profile to user-provided digital images of vehicles comprisingcamera and lighting characteristics of the user-provided digital imageof the vehicle.

In a sixth aspect, in the computer system of any of the first throughfifth aspects, the executable instructions include instructions that areexecutable to configure the computer system to: identify that theclosest match falls outside a predetermined threshold; providing a useran indication that the vehicle was likely repainted. In a seventhaspect, in the computer system of any of the first through sixthaspects, the executable instructions include instructions that areexecutable to configure the computer system to identify at least onerepair area within the user-provided digital image. In an eighth aspect,in the computer system of the seventh aspect, the step of identifying atleast one repair area within the user-provided digital image comprises:detecting the at least one repair area where the mapped at least oneconforming vehicle template differs from the vehicle; accessing, withina repair template database, one or more repair templates; and mapping atleast one conforming repair template to the at least one repair areawithin the user-provided digital image.

In a ninth aspect, in the computer system of the seventh through eighthaspects, the executable instructions include instructions that areexecutable to configure the computer system to: parse the at least onerepair area from the user-provided digital image; and create a modifieduser-provided digital image by replacing the parsed at least one repairarea with visual data from the at least one conforming vehicle template.In a tenth aspect, in the computer system of any of the first throughninth aspects, the executable instructions include instructions that areexecutable to configure the computer system to identify image textwithin the user-provided digital image. In an eleventh aspect, in thecomputer system of any of the first through tenth aspects, the imageprocessing module can include a machine learning algorithm. In a twelfthaspect, in the computer system of any of the first through eleventhaspects, the executable instructions include instructions that areexecutable to configure the computer system to: receive through thenetwork connection, user-derived audio comprising one or more vehiclecharacteristics; and translate the user-derived audio to machine-encodedtext.

In another configuration of the present invention, a computerized methodfor use on a computer system including one or more processors and one ormore computer-readable media having stored thereon executableinstructions that when executed by the one or more processors configurethe computer system to perform a method of identifying coating colorsusing a digital image, for instance on a computer system as defined inthe first through twelfth aspects, the method can include: receiving,through the network connection, a user-provided digital image of avehicle; accessing, within a vehicle template database, one or morevehicle templates; mapping at least one conforming vehicle template tothe vehicle within the user-provided digital image, wherein the at leastone conforming vehicle template can include associated metadatacomprising the one or more vehicle characteristics, including one ormore associated color codes; identifying, with an image processingmodule, a color value associated with the vehicle within theuser-provided digital image, wherein the color value is an RGB value;calculating a closest match for identified color value associated withthe vehicle from the one or more associated color codes; and providing auser the calculated closest match.

In a fourteenth aspect, in the computerized method of the thirteenthaspect, the image processing module is configured to determine the colorof the vehicle in the user-provided digital image and map acolor-matched conforming vehicle template to the vehicle, wherein the atleast one color-matched conforming vehicle template can includeassociated metadata comprising the one or more vehicle characteristics.In a fifteenth aspect, in the computerized method of any of thethirteenth to fourteenth aspects, the method can further includeproviding a user the associated metadata comprising the one or morevehicle characteristics. In a sixteenth aspect, the computerized methodof any of the thirteenth through fourteenth aspects can further includereceiving user feedback that the calculated closest match is incorrect;calculating a color shift profile; and applying the color shift profileto user-provided digital images of vehicles comprising camera andlighting characteristics of the user-provided digital image of thevehicle. In a seventeenth aspect, the computerized method of any of thethirteenth through sixteenth aspects can further include identifyingthat the closest match falls outside a predetermined threshold; andproviding the user an indication that the vehicle was likely repainted.

In an eighteenth aspect, the computerized method of any of thethirteenth through seventeenth aspect can further include identifying atleast one repair area within the user-provided digital image. In thecomputerized method of the eighteenth aspect, the step of identifying atleast one repair area within the user-provided digital image can includedetecting the at least one repair area where the mapped at least oneconforming vehicle template differs from the vehicle; accessing, withina repair template database, a database subset of one or more repairtemplates; and mapping at least one conforming repair template to the atleast one repair area within the user-provided digital image. In atwentieth aspect, the computerized method of any of the thirteenththrough nineteenth aspects, the executable instructions includeinstructions that are executable to configure the computer system to:parse the at least one repair area from the user-provided digital image;and create a modified user-provided digital image by replacing theparsed at least one repair area with visual data from the at least oneconforming vehicle template.

In a twenty-first aspect, in the computerized method of any of thethirteenth to twentieth aspects, the step of identifying, with the imageprocessing module, the one or more vehicle characteristics within theuser-provided digital image can include: identifying image text withinthe user-provided digital image; and translating the identified imagetext to machine-encoded text using optical character recognitiontechnology.

In another configuration, a twenty-second aspect of the invention caninclude a computer program product that includes one or more computerstorage media having stored thereon computer-executable instructionsthat, when executed at a processor, cause a computer system to perform amethod for identify coating colors using a digital image, as inparticular performing the computerized method according to any of thethirteenth through twenty-first aspects, for instance on a computersystem as defines in the first to twelfth aspects, the method caninclude receiving, through a network connection, a user-provided digitalimage of a vehicle; accessing, within a vehicle template database, oneor more vehicle templates; mapping at least one conforming vehicletemplate to the vehicle within the user-provided digital image, whereinthe at least one conforming vehicle template can include associatedmetadata comprising the one or more vehicle characteristics, includingone or more associated color codes; identifying, with an imageprocessing module, a color value associated with the vehicle within theuser-provided digital image; calculating a closest match for identifiedcolor value associated with the vehicle from the one or more associatedcolor codes; providing a user the calculated closest match; receivinguser feedback that the calculated closest match is incorrect;calculating a color shift profile; and applying the color shift profileto user-provided digital images of vehicles comprising camera andlighting characteristics of the user-provided digital image of thevehicle.

1. A computer system for identifying coating colors using a digitalimage, comprising: one or more processors; and one or morecomputer-readable media having stored thereon executable instructionsthat when executed by the one or more processors configure the computersystem to perform at least the following: receive, through a networkconnection, a user-provided digital image of a vehicle; access, within avehicle template database, one or more vehicle templates; map at leastone conforming vehicle template to the vehicle within the user-provideddigital image, wherein the at least one conforming vehicle templatecomprises associated metadata comprising the one or more vehiclecharacteristics, including one or more associated color codes; identify,with an image processing module, a color value associated with thevehicle within the user-provided digital image; and calculate a closestmatch for identified color value associated with the vehicle from theone or more associated color codes.
 2. The computer system of claim 1,wherein the color value associated with the vehicle is an RGB value. 3.(canceled)
 4. The computer system of claim 1, wherein the executableinstructions include instructions that are executable to configure thecomputer system to provide a user the associated metadata comprising theone or more vehicle characteristics.
 5. The computer system of claim 1,wherein the executable instructions include instructions that areexecutable to configure the computer system to: receive user feedbackthat the closest match is incorrect; calculate a color shift profile;and apply the color shift profile to user-provided digital images ofvehicles comprising camera and lighting characteristics of theuser-provided digital image of the vehicle.
 6. The computer system ofclaim 1, wherein the executable instructions include instructions thatare executable to configure the computer system to: identify that theclosest match falls outside a predetermined threshold; and provide auser an indication that the vehicle was likely repainted.
 7. Thecomputer system of claim 1, wherein the executable instructions includeinstructions that are executable to configure the computer system toidentify at least one repair area within the user-provided digitalimage.
 8. The computer system of claim 7, wherein identifying at leastone repair area within the user-provided digital image comprises:detecting the at least one repair area where the mapped at least oneconforming vehicle template differs from the vehicle; accessing, withina repair template database, one or more repair templates; and mapping atleast one conforming repair template to the at least one repair areawithin the user-provided digital image.
 9. The computer system of claim7, wherein the executable instructions include instructions that areexecutable to configure the computer system to: parse the at least onerepair area from the user-provided digital image; and create a modifieduser-provided digital image by replacing the parsed at least one repairarea with visual data from the at least one conforming vehicle template.10. The computer system of claim 1, wherein the executable instructionsinclude instructions that are executable to configure the computersystem to identify image text within the user-provided digital image.11. The computer system of claim 1, wherein the image processing modulecomprises a machine learning algorithm.
 12. The computer system of claim1, wherein the executable instructions include instructions that areexecutable to configure the computer system to: receive through thenetwork connection, user-derived audio comprising one or more vehiclecharacteristics; and translate the user-derived audio to machine-encodedtext.
 13. A computerized method for use on a computer system comprisingone or more processors and one or more computer-readable media havingstored thereon executable instructions that when executed by the one ormore processors configure the computer system to perform a method ofidentifying coating colors using a digital image, the method comprising:receiving, through the network connection, a user-provided digital imageof a vehicle; accessing, within a vehicle template database, one or morevehicle templates; mapping at least one conforming vehicle template tothe vehicle within the user-provided digital image, wherein the at leastone conforming vehicle template comprises associated metadata comprisingthe one or more vehicle characteristics, including one or moreassociated color codes; identifying, with an image processing module, acolor value associated with the vehicle within the user-provided digitalimage; calculating a closest match for identified color value associatedwith the vehicle from the one or more associated color codes; andproviding a user the calculated closest match.
 14. (canceled)
 15. Thecomputerized method of claim 13, further comprising providing a user theassociated metadata comprising the one or more vehicle characteristics.16. The computerized method of claim 13, further comprising: receivinguser feedback that the calculated closest match is incorrect;calculating a color shift profile; and applying the color shift profileto user-provided digital images of vehicles comprising camera andlighting characteristics of the user-provided digital image of thevehicle.
 17. The computerized method of claim 13, further comprising:identifying that the closest match falls outside a predeterminedthreshold; and providing the user an indication that the vehicle waslikely repainted.
 18. The computerized method of claim 13, furthercomprising identifying at least one repair area within the user-provideddigital image.
 19. The computerized method of claim 18, whereinidentifying at least one repair area within the user-provided digitalimage comprises: detecting the at least one repair area where the mappedat least one conforming vehicle template differs from the vehicle;accessing, within a repair template database, a database subset of oneor more repair templates; and mapping at least one conforming repairtemplate to the at least one repair area within the user-provideddigital image.
 20. The computerized method of claim 13, wherein theexecutable instructions include instructions that are executable toconfigure the computer system to: parse the at least one repair areafrom the user-provided digital image; and create a modifieduser-provided digital image by replacing the parsed at least one repairarea with visual data from the at least one conforming vehicle template.21. The computerized method of claim 13, wherein identifying, with theimage processing module, the one or more vehicle characteristics withinthe user-provided digital image comprises: identifying image text withinthe user-provided digital image; and translating the identified imagetext to machine-encoded text using optical character recognitiontechnology.
 22. A computer program product comprising one or morecomputer storage media having stored thereon computer-executableinstructions that, when executed at a processor, cause a computer systemto perform a method for identify coating colors using a digital image,the method comprising: receiving, through a network connection, auser-provided digital image of a vehicle; accessing, within a vehicletemplate database, one or more vehicle templates; mapping at least oneconforming vehicle template to the vehicle within the user-provideddigital image, wherein the at least one conforming vehicle templatecomprises associated metadata comprising the one or more vehiclecharacteristics, including one or more associated color codes;identifying, with an image processing module, a color value associatedwith the vehicle within the user-provided digital image; calculating aclosest match for identified color value associated with the vehiclefrom the one or more associated color codes; providing a user thecalculated closest match; receiving user feedback that the calculatedclosest match is incorrect; calculating a color shift profile; andapplying the color shift profile to user-provided digital images ofvehicles comprising camera and lighting characteristics of theuser-provided digital image of the vehicle.